Evaluating epidemic forecasts in an interval format

For practical reasons, many forecasts of case, hospitalization and death counts in the context of the current COVID-19 pandemic are issued in the form of central predictive intervals at various levels. This is also the case for the forecasts collected in the COVID-19 Forecast Hub (https://covid19forecasthub.org/). Forecast evaluation metrics like the logarithmic score, which has been applied in several infectious disease forecasting challenges, are then not available as they require full predictive distributions. This article provides an overview of how established methods for the evaluation of quantile and interval forecasts can be applied to epidemic forecasts in this format. Specifically, we discuss the computation and interpretation of the weighted interval score, which is a proper score that approximates the continuous ranked probability score. It can be interpreted as a generalization of the absolute error to probabilistic forecasts and allows for a decomposition into a measure of sharpness and penalties for over- and underprediction

Time series analysis of malaria in Afghanistan: using ARIMA models to predict future trends in incidence

Additional file 6: Annex 1. Right side: Autocorrelation (ACF) and partial autocorrelation (PACF) functions of the residuals from ARIMA model (1, 0, 1) × (1, 0, 1)12 on log-transformed, differenced data. Left side: ACF and PACF of the residuals from ARIMA model (4, 0, 1) × (1, 0, 1)12 on log-transformed, differenced data

Epidemic dynamics of respiratory syncytial virus in current and future climates.

A key question for infectious disease dynamics is the impact of the climate on future burden. Here, we evaluate the climate drivers of respiratory syncytial virus (RSV), an important determinant of disease in young children. We combine a dataset of county-level observations from the US with state-level observations from Mexico, spanning much of the global range of climatological conditions. Using a combination of nonlinear epidemic models with statistical techniques, we find consistent patterns of climate drivers at a continental scale explaining latitudinal differences in the dynamics and timing of local epidemics. Strikingly, estimated effects of precipitation and humidity on transmission mirror prior results for influenza. We couple our model with projections for future climate, to show that temperature-driven increases to humidity may lead to a northward shift in the dynamic patterns observed and that the likelihood of severe outbreaks of RSV hinges on projections for extreme rainfall

High turnover drives prolonged persistence of influenza in managed pig herds.

Pigs have long been hypothesized to play a central role in the emergence of novel human influenza A virus (IAV) strains, by serving as mixing vessels for mammalian and avian variants. However, the key issue of viral persistence in swine populations at different scales is ill understood. We address this gap using epidemiological models calibrated against seroprevalence data from Dutch finishing pigs to estimate the 'critical herd size' (CHS) for IAV persistence. We then examine the viral phylogenetic evidence for persistence by comparing human and swine IAV. Models suggest a CHS of approximately 3000 pigs above which influenza was likely to persist, i.e. orders of magnitude lower than persistence thresholds for IAV and other acute viruses in humans. At national and regional scales, we found much stronger empirical signatures of prolonged persistence of IAV in swine compared with human populations. These striking levels of persistence in small populations are driven by the high recruitment rate of susceptible piglets, and have significant implications for management of swine and for overall patterns of genetic diversity of IAV.This work was supported by the RAPIDD program of the Science and Technology Directorate, Department of Homeland Security, and the Fogarty International Center, National Institutes of Health (V.E.P., S.R., J.L.N.W. and B.T.G.) and the Bill & Melinda Gates Foundation (V.E.P. and B.T.G.). J.L.N.W. is also supported by the Alborada Trust, the European Union FP7 project ANTIGONE (contract no. 278976) and by Biotechnology and Biological Sciences Research Council sLOLA BB/L001330/1.This is the final version of the article. It first appeared from Royal Society Publishing via http://dx.doi.org/10.1098/rsif.2016.013

Calibration of individual-based models to epidemiological data : a systematic review

Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy–either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures

Predicting the Impact of Vaccination on the Transmission Dynamics of Typhoid in South Asia: A Mathematical Modeling Study

Background: Modeling of the transmission dynamics of typhoid allows for an evaluation of the potential direct and indirect effects of vaccination; however, relevant typhoid models rooted in data have rarely been deployed. Methodology/Principal Findings We developed a parsimonious age-structured model describing the natural history and immunity to typhoid infection. The model was fit to data on culture-confirmed cases of typhoid fever presenting to Christian Medical College hospital in Vellore, India from 2000–2012. The model was then used to evaluate the potential impact of school-based vaccination strategies using live oral, Vi-polysaccharide, and Vi-conjugate vaccines. The model was able to reproduce the incidence and age distribution of typhoid cases in Vellore. The basic reproductive number (R0) of typhoid was estimated to be 2.8 in this setting. Vaccination was predicted to confer substantial indirect protection leading to a decrease in the incidence of typhoid in the short term, but (intuitively) typhoid incidence was predicted to rebound 5–15 years following a one-time campaign. Conclusions/Significance: We found that model predictions for the overall and indirect effects of vaccination depend strongly on the role of chronic carriers in transmission. Carrier transmissibility was tentatively estimated to be low, consistent with recent studies, but was identified as a pivotal area for future research. It is unlikely that typhoid can be eliminated from endemic settings through vaccination alone

Little Evidence for Genetic Susceptibility to Influenza A (H5N1) from Family Clustering Data

The apparent clustering of human cases of influenza A (H5N1) among blood relatives has been considered as evidence of genetic variation in susceptibility. We show that, by chance alone, a high proportion of clusters are expected to be limited to blood relatives when infection is a rare event